The great performance of deep learning in many visual tasks largely depends on the big data volume and the improvement of computing power. But in many practical projects, it is usually difficult to provide enough data to complete the task. Concerning the problem that the number of infrared images is small and the infrared images are hard to collect, a method to generate infrared images based on color images was proposed to obtain more infrared image data. Firstly, the existing color image and infrared image data were employed to construct the paired datasets. Secondly, the generator and the discriminator of Generative Adversarial Network (GAN) model were formed based on the convolutional neural network and the transposed convolutional neural network. Thirdly, the GAN model was trained based on the paired datasets until the Nash equilibrium between the generator and the discriminator was reached. Finally, the trained generator was used to transform the color image from the color field to the infrared field. The experimental results were evaluated based on quantitative evaluation metrics. The evaluation results show that the proposed method can generate high-quality infrared images. In addition, after the L1 or L2 regularization constraint was added to the loss function, the FID (Fréchet Inception Distance) score was respectively reduced by 23.95, 20.89 on average compared to the FID score of loss function not adding the constraint. As an unsupervised data augmentation method, the method can also be applied to many other visual tasks that lack train data, such as target recognition, target detection and data imbalance.